Discussion : “ Factors of Safety for

A recent paper by Xing and Stern [1] provides additional performance data of interest on Verification of Calculations, but is misleading in some important respects. Five methods, all starting from Richardson Extrapolation, are compared: three are based on variants of my Grid Convergence Index method [2–6], and two are based on the authors’ work: the earlier “CF method” and the latest “FS method.” (1) The method referred to as GCI2 is successful but its source is cited as a “private communication” from myself (their Ref. [16]). What they refer to as GCI2 is essentially the GCI method, as I have described and discussed and evaluated at length, Refs. [2–6]. This is the method that, as the authors say [1], “is widely used and recommended by ASME and AIAA” citing [6,7]. I shall refer to it herein as the “real GCI method” for reasons soon to be apparent. The equivalence is obscured because the authors [1] have written their Eq. (12) defining the method using their own ratio term CF, which obscures the simplicity of the original and makes it appear that the factor of safety varies continuously with observed order of convergence pRE; see Fig. 3 of Ref. [1]. This is merely an artifact of Eq. (12) in which their factor of safety FS is defined in terms of an error estimate based on observed pRE rather than on the p actually used. As a result, the “factor of safety” FS in Ref. [1] is not always the same concept as the “factor of safety” FS in Refs. [2–6]. This leaves the question of what are the methods called “GCI” and “GCI1” by the authors. The answer is that I consider these to be misapplications of the GCI formulas, and therefore misrepresentations of the GCI method. (2) For specificity, I will change notation and refer to the method of their Eq. (10) as GCI0, not as “GCI” since I claim it is not the real GCI method. This Eq. (10) is written in terms of an error estimate based on the “observed” [2–6] or “estimated” [1] order of convergence pRE. However, the authors correctly indicate in the text following that this is possible only if three grids have been used to calculate a pRE, in which case the recommended Factor of Safety FS1⁄4 1.25. If only two grids have been used, pRE in Eq. (10) is replaced by a theoretical value pth (e.g., pth1⁄4 2 for a nominally second-order method) and a more conservative value FS1⁄4 3 is recommended [2–6]. This would indeed agree with my GCI method [2–6] in good applications. The trouble is that the authors have applied this regardless of the reasonableness of the estimated pRE. This is contrary to the discussions in Ref. [2], which discussions I claim are part of the real “GCI method” as contrasted with simplistic application of the formula. Exactly what constitutes a reasonable value was not defined in my work [2–6] and I would not take issue with (say) a 5% difference between pRE and pth, e.g., using an observed pRE1⁄4 2.1 for a nominally second-order method. (Although it should be obvious that the conservative approach would be to use the minimum of pRE and pth, I would not claim that this was a misapplication of the GCI method.) However, the authors [1] have applied the GCI0 formula using observed pRE theoretical pth, which is imprudent and not recommended [2–6] no matter what the value of Fs. This point was conveyed to the authors of Ref. [1] in their Ref. [16], the private communication from myself. (I am glad to make this communication, cited here as Ref. [8], available to any interested party.) Using the authors’ notation P1⁄4 pRE/pth the authors have claimed rational estimates of uncertainty from studies with P as high as 6.01 for the new ship hydrodynamics problem (see Table 7, second line). Since the numerical method used is nominally second order, this indicates an observed order pRE1⁄4 12.02, the use of which would be ridiculous. Another instance is the third row entry in Table 6 [1] for the grid triplet (4,5,6) showing P1⁄4 0.08. These are pretty strong hints that something is wrong. There is no sense in performing a grid convergence study and then ignoring the results. No excuses of the computing difficulties in industrial applications can justify an attempt to obtain a rational estimate of uncertainty from such meaningless corrupted studies. To so attempt is to give merit to meritless results. I can only describe this as a misapplication of the GCI formula, not as a poor result for the real GCI method. (3) The second method considered in Ref. [1] is GCI1 based on Eq. (11). The appearance of the CF term in the equation serves to exchange the error estimate based on observed pRE to one based on theoretical pth for the condition P >1, i.e., for pRE> pth. This is exactly what is recommended above, and is good conservative practice. However, the Eq. (11) uses Fs1⁄4 1.25 in either case. This does not make sense. The GCI1 method as described uses three grid solutions to evaluate an observed pRE, but if this value is too large, the method reverts to using exactly the error estimator that would have been used for only two-grid studies but with the nonconservative value Fs1⁄4 1.25 instead of the recommended Fs1⁄4 3. Thus, the authors did use the correct factor of safety Fs1⁄4 3 as recommended [2 6] for 2 grid studies but they used only Fs1⁄4 1.25 for 3-grid studies in which the observed order of convergence pRE was not at least approximately consistent with theory. The authors then concluded from their tests that “These facts suggest that the use of the GCI1 method is closer to a 68% than a 95% confidence level.” This is a misleading statement since the conclusion follows from using Fs1⁄4 1.25 and applying it to a dataset from [9] designed with “intentional choice of grid studies with oscillations in both exponent p and output quantity” [9] (Ref. 3). The point of such studies as Ref. [1] should not be to tune large Fs until 95% coverage is obtained no matter how bad the data set is. If such large Fs are required, this should be a flag alerting the investigator that the grids used are inadequate for the problem [3,10,11]. (4) Therefore, the evaluation in Ref. [1] of the real GCI method (GCI2) is corrupted by confusion with two distorted methods. The authors’ evaluation also does not agree with my own evaluation from some of the same data, notably the wide-ranging study of Cadafalch et al. [12] (Ref. [33] of Ref. [1]) which I presented in Contributed by the Fluids Engineering Division of ASME for publication in the JOURNAL OF FLUIDS ENGINEERING. Manuscript received Ocotber 19, 2010; final manuscript received July 28, 2011; published online October 24, 2011. Assoc. Editor: Dimitris Drikakis. In the original GCI method [2], FS does depend on observed pRE implicitly, in the sense that if pRE is unreasonable one at least reverts to using pth and FS1⁄4 3 rather than blindly applying FS1⁄4 1.25. The preferred approach would be to continue the grid convergence study with additional grids until reasonable values of pRE were obtained. These rather obvious points have been made more explicit in Ref. [3] which was not available to the authors of Ref. [1]. There is further unfortunate confusion caused by the authors’ choice of a descriptor for their method and their use of the symbol FS both for their “Factor of Safety” method and for the factors of safety used in all five methods, including the “factor of safety used in the Factor of Safety method.” Likewise for their symbol CF, which denotes both their Correction Factor method and the “correction factor” used in the Correction Factor method as well as in their defining equations for other methods (Eqs. (11) and (12)).